1. Field of the Invention
The present invention relates to a three-dimensional image processing apparatus, a three-dimensional image processing method, a three-dimensional image processing program, a computer-readable recording medium, and a recording device.
2. Description of Related Art
In a large number of production sites such as factories, there have been introduced image processing apparatuses that realize automatic and fast performance of inspections, which have relied on viewing of humans. The image processing apparatus captures an image of a workpiece that comes flowing through a production line such as a belt conveyor by use of a camera, and executes measurement processing such as edge detection and area calculation for a predetermined region by use of the obtained image data. Then, based on a processing result of the measurement processing, the apparatus performs inspections such as detection of a crack on the workpiece and positional detection of alignment marks, and outputs determination signals for determining the presence or absence of a crack on the workpiece and positional displacement. In such a manner, the image processing apparatus may be used as one of FA (Factory Automation) sensors.
An image which is taken as a measurement processing target by the image processing apparatus that is used as the FA sensor is principally a brightness image not including height information. For this reason, speaking of the foregoing detection of a crack on the workpiece, the apparatus is good at stably detecting a two-dimensional shape of a cracked portion, but having difficulties in stably detecting a three-dimensional shape of, for example, a depression of a flaw which is not apt to appear in a brightness image. For example, it is thought that a type or a direction of illumination that illuminates the workpiece during the inspection is devised and a shade caused by a depression of a flaw is detected to indirectly detect a three-dimensional shape, but a clear shade is not necessarily always detected in the brightness image. In order to prevent an erroneous determination which is to erroneously detect a defective product as a non-defective product at the time of an unclear shade being detected, for example, if a determination threshold is biased to the safe side, the apparatus might determine a large number of non-defective products as defective products, to cause deterioration in production yield.
Accordingly, there is considered a visual inspection which uses not only a shade image that takes, as a pixel value, a shade value in accordance with a light reception amount of the camera but also a distance image that takes, as a pixel value, a shade value in accordance with a distance from the camera to the workpiece to two-dimensionally express a height (e.g. see Unexamined Japanese Patent Publication No. 2012-21909).
An example of the three-dimensional image processing apparatus is shown in a schematic view of FIG. 23. This three-dimensional image processing apparatus 190 is configured of a head section 191 provided with an image capturing part such as a light reception element, and a controller section 192 which is connected to the head section 191 and is sent image data captured in the head section 191, to generate a distance image from the acquired image data.
Here, the principle of triangulation will be described based on FIG. 23. In the head section 191, an angle α between an optical axis of incident light emitted from a light projecting section 110 and an optical axis of reflected light that is incident on a light receiving section 120 (optical axis of the light receiving section 120) is previously set. Here, when the workpiece WK is not mounted on a stage 140, incident light emitted from the light projecting section 110 is reflected by a point O on the workpiece WK mounting surface and is incident on the light receiving section 120. On the other hand, when the workpiece WK is mounted on the stage 140, the incident light emitted from the light projecting section 110 is reflected by a point A on the surface of the workpiece WK and is incident as reflected light on the light receiving section 120. Then, a distance d in an X-direction between the point O and the point A is measured, and based on this distance d, a height h of the point A on the surface of the workpiece WK is calculated.
Heights of all points on the surface of the workpiece WK are calculated applying the foregoing the measurement principle of triangulation, thereby to measure a three-dimensional shape of the workpiece WK. In a pattern projecting method, in order that all the points on the surface of the workpiece WK are irradiated with incident light, the incident light is emitted from the light projecting section 110 in accordance with a predetermined structured pattern, reflected light as the light reflected on the surface of the workpiece WK is received, and based on a plurality of received pattern images, the three-dimensional shape of the workpiece WK is efficiently measured.
As such a pattern projecting method, there are known a phase shift method, a spatial coding method, a multi-slit method and the like. By the three-dimensional measurement processing performed using the pattern projecting method, a projection pattern is changed to repeat image-capturing a plurality of times in the head section, and the images are transmitted to the controller section. In the controller section, computing is performed based on the pattern projected images transmitted from the head section, and a distance image having height information of the workpiece can be obtained.
In 3D measurement using the phase shift method and the spatial coding method by the configuration of the camera and the projector as thus described, a fringe pattern as shown in FIG. 24 is projected as a projection pattern PN. However, in the pattern projecting method, there has been a problem in that a measurement result which is incorrect in principle is obtained in a place where multi-reflection occurs. As shown in FIG. 25, especially on the side surface of the workpiece WK which stands upright as seen from the camera, measurement becomes incorrect due to an influence of mirror reflection, and post-stage processing is affected by mixture of an effective result and an incorrect result.
However, since an image of a mirror-reflected pattern is captured as a clear pattern, differently from the case of deficiency in light amount, it is displayed as a clear measurement result similar to a normal measurement result. It has hitherto been difficult to determine whether the result is abnormal and disadvantageously a different result from the original has been obtained.
Specifically, as in FIG. 26, there will be considered an example where an image of workpieces WKA, WKB and WKC placed on a stage ST for placing the workpiece WK is captured by means of a camera as an image capturing part 10 and a projector as a light projecting part 20. In this case, the workpieces WKA, WKB and WKC as seen from the camera are as in FIG. 27. In this state, when a projection pattern PN0 as in FIG. 24 is projected to the workpieces WKA, WKB and WKC, an image of a fringe pattern captured in the camera is as in FIG. 28. Among them, the fringe patterns obtained by capturing the workpiece WKB and the workpiece WKC are as in an enlarged view of FIG. 29. Here, since the side surfaces of the workpiece WKB and the workpiece WKC have acute angles as shown in FIG. 25, each of them comes into a state where a pattern on the bottom surface is reflected thereon, namely the mirror reflection occurs (in FIG. 29, foreign substances are displayed for facilitating understanding of reflection of mirror patterns due to the mirror reflection). As a result, there is obtained a pattern different from a pattern that should originally be obtained in a state where the mirror reflection does not occur. Specifically, an original distance image to be measured with respect to the workpieces of FIG. 27 should be as in FIG. 30A, but as a result of capturing of the image of the fringe pattern as in FIG. 28, an incorrect distance image as in FIG. 30B is measured. It is to be noted that in these drawings, a region which is shaded by the workpiece and whose height is thus non-measurable is shown by being painted out in black. Further, a brighter place (a place closer to white) indicates it has a larger height, and a darker place indicates it has a smaller height.
As apparent by comparing FIG. 30A and FIG. 30B, a normal height has been measured concerning the workpiece WKA. On the other hand, as for the workpiece WKB, the top surface has been correctly measured since no mirror reflection occurs, but the front-side surface has been computed as incorrect height information as if it is continuous with the floor surface of the stage ST at the same height. Further, as for the workpiece WKC, the top surface has been correctly measured as well, but an incorrect measurement result has been obtained for the side surface facing lower left as if it further goes under the floor surface of the stage.
The reason why such incorrect results are computed is that, as a result of occurrence of the mirror reflection, a projection pattern different from a projection pattern that should originally be obtained is measured and a result of height calculation by means of triangulation thus becomes incorrect. Specifically, in the case of the workpiece WKB, as shown in FIG. 31C, the fringe pattern is projected from an oblique direction by the projector to the side surface of the workpiece WKB, and hence height information is acquired at an intersection with a visual line of an observation point. When such a fringe pattern is projected to the workpiece WKB, an original normal projection pattern is as in FIG. 31A. However, as a result of occurrence of the mirror reflection, a pattern on the floor surface of the stage is reflected on the side surface, and a projection pattern as in FIG. 31B is obtained. For example, when a focus is placed on an observation point P1 on the side surface of FIG. 31A, it should originally be irradiated with a fringe 3, but it actually looks as if being irradiated with a fringe 2 as in FIG. 31B. Consequently, as in FIG. 31C, height information of the side surface of the workpiece WKB is detected not as one at an intersection of the visual line of the observation point with the fringe 3 which has an original height, but as one at an intersection with the fringe 2, resulting in that the side surface is measured as having the same height as that of the floor surface of the stage.
Further, in the case of the workpiece WKC, as shown in FIG. 32C, when the fringe pattern is projected from an oblique direction from the projector to the side surface of the workpiece WKC, an original normal projection pattern should be as in FIG. 32A, but a projection pattern as in FIG. 32B is obtained due to the mirror reflection. In this example, when a focus is placed on an observation point P2, it should originally be irradiated with a fringe 5, but it actually looks as if being irradiated with a fringe 4 as in FIG. 32B. Consequently, as in FIG. 32C, height information of the side surface of the workpiece WKC is detected not as one at an intersection of a visual line of the observation point with the fringe 5 which has an original height, but as one at an intersection with the fringe 4, resulting in that the side surface is measured as having a smaller height than that of the floor surface of the stage.
As thus described, in a portion where the mirror reflection has occurred, it is not possible to distinguish between a normal measurement result and an incorrect measurement result, thus causing a problem of affecting the post-stage processing.